Attribute assisted teacher-critical training strategies for image captioning

Existing image captioning models are usually trained by cross-entropy (XE) loss and reinforcement learning (RL), which set ground-truth words as hard targets force the model to learn from them. However, widely adopted training strategies may suffer misalignment in XE inappropriate reward assignment RL training. To tackle these problems, we introduce an attribute enhanced teacher that serves a bridge between captions generating some easier-to-learn word proposals soft targets. Currently, most knowledge distillation methods build introducing more parameters well additional data. In our proposal, alternatively construct utilizing attributes already exist can be very easily extracted. effectively model, further propose Teacher-Critical Training Strategies (TCTS) for both facilitate efficient of model. Experimental evaluations several architectures on benchmark MSCOCO dataset show proposed TCTS comprehensively outperforms baselines subjective metrics human evaluations. Our codes pre-trained will open-sourced.